Language trends after large-scale upheavals - SPSP companion page

How do you define mass shootings?

Our data source, The Violence Project, defines a public mass shooting as follows:

“a multiple homicide incident in which four or more victims are murdered with firearms—not including the offender(s)—within one event, and at least some of the murders occurred in a public location or locations in close geographical proximity (e.g., a workplace, school, restaurant, or other public settings), and the murders are not attributable to any other underlying criminal activity or commonplace circumstance (armed robbery, criminal competition, insurance fraud, argument, or romantic triangle).”

Which mass shootings did you include?

We included all 60 mass shootings in the United States from April 2013 to March 2023 (The Violence Project). You can learn more about them in the table below. Click on a column title to sort by it.

use_subreddit = name of the subreddit used for the analysis. n_posts = number of Reddit comments containing at least 15 words. n_authors = number of unique authors. min_date = date of earliest Reddit post used (set to 6 weeks before the event). max_date = date of latest Reddit post used (set to 6 weeks after the event).

Below is a map of the mass shootings included in our analysis. Each circle is a mass shooting, and its radius is proportional to the number of people killed and the number of people injured.

What are federally declared natural disasters?

When a disaster hits a state, the governor of said state can request for a declaration by the President. The declaration paves way for the state to receive federal assistance in recovering from the disaster. For more information on how disasters get declared, check out this video from Federal Emergency Management Agency.

Which disasters did you include?

We included all 196 federally declared disasters in the United States from April 2013 to March 2023 (FEMA).

In cases where a disaster was declared both as an emergency and a major disaster (these are the two declaration types), we only kept one observation.

Since each disaster declaration is filed by a state, you may see the same disaster multiples times if multiple states filed requests.

You can learn more about the disasters included in the table below. Click on a column title to sort by it.

use_subreddit = name of the subreddit used for the analysis. n_posts = number of Reddit comments containing at least 15 words. n_authors = number of unique authors. min_date = date of earliest Reddit post used (set to 6 weeks before the event). max_date = date of latest Reddit post used (set to 6 weeks after the event).

This map might help you situate the disasters. Each circle is a disaster, and its radius is proportional to the dollar amount of federal grant given to assist in recovery from the disaster.

What is Linguistic Inquiry and Word Count (LIWC)? Are there other ways to detect language trends?

LIWC is a bag of word approach to studying language. In a nutshell, it counts words that belong to a dictionary in a given piece of text. These dictionaries are calibrated to correspond to elusive things like cognitive processing and emotions. You can read more about it on the (LIWC website)[https://www.liwc.app]. But as you know, these elusive things are, well, elusive, and the dictionaries aren’t always the best bet.

In our case, though, we have opted for LIWC because of its simplicity, which makes interpretation of the results somewhat more intuitive. Each measure (except for Analytic, which is a composite score) is literally the % of words in the given text that are in the dictionary.

There certainly are many other ways to look at language. If you think there’s an approach that suits our use case, please

So what words are in the dictionaries you used?

Check out the LIWC user manual!

Not all upheavals are created equal… have you thought about how variations in the severity of the upheavals may affect your results?

We think this is a super important point! In fact, we tried to redo our analysis using some proxies for severity. The results in general look like what you would predict: more severe upheavals saw larger effects. But there are exceptions. Check them out for yourself in the plots below.

Which severity proxies did you use?

For mass shootings, we used the number of people killed + the number of people injured. For disasters, we used the total amount of federal grant given to an affected region.

These are admittedly very crude proxies. For example, the number of people affected by the disasters would be a better way to make the proxy for disasters comparable to that for the mass shootings. The proxies we used also don’t really get at how severe people think these upheavals are. We think perceived severity probably affects language use more than actual severity. It may be better captured by something like the amount of media coverage/some measure of how much people talked about the upheavals on-/off-line.

Can I look explore the language trends for each upheaval separately?

We’re working on a more user-friendly way to present this. Coming soon!

I have a better proxy/way of running this analysis in mind!

We would be so appreciative if you could share it in this form!

Show me the plots!

Mass shootings

Disasters

What about time? You cover a 10-year span in your analysis. Maybe something has changed over time in how people react to these upheavals…

We think you’re totally onto something! Just anecdotally, we have found seeing distressing news every day making us less able/willing to grasp just how distressing these events actually are. Overtime we seem to be slowly but surely being desensitized to things that should have drawn out more reactions from us.

We broke down the analysis into three time bins in the plots below. This is obviously super crude, and we’d love to hear that better idea forming in your head right now: please share it in this (form)[URL NEEDED]!

Show me the plots!

Mass shootings

Disasters

I like plots and stuff, but you haven’t run any statistical tests? How can I know if this is true?

Ha yes—in this poster we’ve erred toward the side of simplicity. You’re absolutely right that your eyes can trick you into seeing something that Does Not Exist. Although your stats can probably do a very good job at that, too.

Anyhow, we ran a few paired t-tests (again doing the simplest thing possible). Here’s how we did it: * We took the mean of linguistic measures from 6 weeks to 2 weeks before the upheaval as baseline. Each author gets their own baseline. * For every week-long period from 1 week before the upheaval to 6 weeks after the upheaval, we compared the author’s linguistic measures in that period to those in their baseline. * If there lacks a statistically significant (we set the bar at the 3% level) and substantially meaningful (well, this is up to your interpretation—we’re again reminding you that all measures, except for Analytic, are actual % of words in the text), then we say that the person has reverted back to their baseline. * Note: week 0 is the day of the upheaval.

There’re certainly better ways to do it! Pplease do share your thoughts in this (form)[URL NEEDED].

t-tests

Shootings

Characteristic	-1 wk, N = 26,9621	baseline, N = 26,9621	Difference2	95% CI2,3	p-value2
i	3.57 (3.48)	3.62 (3.01)	-0.05	-0.10, 0.00	0.042
we	0.54 (1.37)	0.52 (1.09)	0.02	0.00, 0.04	0.031
they	1.37 (2.09)	1.40 (1.75)	-0.02	-0.05, 0.01	0.2
Analytic	47 (26)	48 (22)	-0.71	-1.1, -0.32	<0.001
cogproc	12.4 (5.9)	12.2 (4.9)	0.17	0.09, 0.26	<0.001
prosocial	0.47 (1.18)	0.46 (0.96)	0.01	-0.01, 0.03	0.2
emo_anx	0.08 (0.47)	0.09 (0.41)	-0.01	-0.01, 0.00	0.15
emo_anger	0.15 (0.64)	0.14 (0.51)	0.01	0.00, 0.02	0.059
emo_sad	0.07 (0.44)	0.07 (0.36)	0.00	-0.01, 0.01	0.6
1 Mean (SD)
2 Paired t-test
3 CI = Confidence Interval

Characteristic	0 wk, N = 25,6711	baseline, N = 25,6711	Difference2	95% CI2,3	p-value2
i	3.54 (3.49)	3.58 (2.97)	-0.05	-0.10, 0.00	0.068
we	0.56 (1.38)	0.52 (1.12)	0.04	0.02, 0.06	<0.001
they	1.39 (2.11)	1.40 (1.73)	-0.02	-0.05, 0.02	0.3
Analytic	47 (26)	48 (22)	-0.52	-0.91, -0.12	0.011
cogproc	12.4 (5.9)	12.2 (4.9)	0.15	0.06, 0.24	0.001
prosocial	0.45 (1.15)	0.45 (0.98)	0.00	-0.02, 0.02	0.9
emo_anx	0.08 (0.45)	0.08 (0.40)	0.00	-0.01, 0.00	0.2
emo_anger	0.15 (0.64)	0.14 (0.53)	0.01	0.00, 0.02	0.020
emo_sad	0.07 (0.45)	0.07 (0.36)	0.00	0.00, 0.01	0.4
1 Mean (SD)
2 Paired t-test
3 CI = Confidence Interval

Characteristic	1 wk, N = 25,8941	baseline, N = 25,8941	Difference2	95% CI2,3	p-value2
i	3.51 (3.44)	3.61 (3.04)	-0.10	-0.15, -0.05	<0.001
we	0.57 (1.38)	0.53 (1.11)	0.05	0.02, 0.07	<0.001
they	1.37 (2.03)	1.39 (1.76)	-0.02	-0.05, 0.01	0.2
Analytic	46 (26)	47 (22)	-1.1	-1.5, -0.68	<0.001
cogproc	12.6 (5.9)	12.2 (4.9)	0.35	0.26, 0.44	<0.001
prosocial	0.49 (1.19)	0.46 (0.98)	0.04	0.02, 0.05	<0.001
emo_anx	0.10 (0.51)	0.08 (0.39)	0.01	0.01, 0.02	0.001
emo_anger	0.16 (0.69)	0.14 (0.54)	0.02	0.01, 0.04	<0.001
emo_sad	0.10 (0.53)	0.07 (0.38)	0.03	0.02, 0.04	<0.001
1 Mean (SD)
2 Paired t-test
3 CI = Confidence Interval

Characteristic	2 wk, N = 23,9331	baseline, N = 23,9331	Difference2	95% CI2,3	p-value2
i	3.61 (3.49)	3.61 (3.01)	0.01	-0.05, 0.06	0.8
we	0.52 (1.32)	0.52 (1.09)	0.00	-0.02, 0.02	0.7
they	1.44 (2.14)	1.38 (1.72)	0.06	0.03, 0.09	<0.001
Analytic	47 (26)	48 (22)	-0.41	-0.82, 0.01	0.054
cogproc	12.4 (5.9)	12.2 (4.9)	0.15	0.06, 0.24	0.002
prosocial	0.46 (1.19)	0.46 (1.00)	0.00	-0.02, 0.02	0.9
emo_anx	0.08 (0.45)	0.08 (0.39)	0.00	-0.01, 0.01	0.6
emo_anger	0.14 (0.61)	0.13 (0.48)	0.01	0.00, 0.02	0.041
emo_sad	0.07 (0.45)	0.07 (0.36)	0.00	-0.01, 0.01	0.6
1 Mean (SD)
2 Paired t-test
3 CI = Confidence Interval

Characteristic	3 wk, N = 23,0131	baseline, N = 23,0131	Difference2	95% CI2,3	p-value2
i	3.57 (3.47)	3.60 (2.99)	-0.03	-0.08, 0.02	0.3
we	0.54 (1.36)	0.52 (1.10)	0.01	-0.01, 0.03	0.3
they	1.44 (2.15)	1.39 (1.76)	0.05	0.02, 0.09	0.002
Analytic	47 (26)	48 (22)	-0.44	-0.86, -0.02	0.041
cogproc	12.3 (5.9)	12.2 (4.9)	0.07	-0.02, 0.17	0.12
prosocial	0.46 (1.16)	0.45 (0.96)	0.01	-0.01, 0.03	0.2
emo_anx	0.08 (0.43)	0.08 (0.39)	-0.01	-0.02, 0.00	0.030
emo_anger	0.13 (0.59)	0.13 (0.50)	0.00	-0.01, 0.01	0.6
emo_sad	0.07 (0.45)	0.07 (0.36)	0.00	-0.01, 0.00	0.4
1 Mean (SD)
2 Paired t-test
3 CI = Confidence Interval

Characteristic	4 wk, N = 22,2311	baseline, N = 22,2311	Difference2	95% CI2,3	p-value2
i	3.52 (3.45)	3.62 (3.02)	-0.10	-0.15, -0.04	<0.001
we	0.51 (1.29)	0.52 (1.09)	-0.01	-0.03, 0.01	0.4
they	1.43 (2.12)	1.39 (1.73)	0.04	0.01, 0.08	0.025
Analytic	48 (26)	48 (22)	-0.15	-0.58, 0.28	0.5
cogproc	12.3 (5.8)	12.2 (4.9)	0.08	-0.01, 0.18	0.086
prosocial	0.46 (1.15)	0.46 (1.05)	0.00	-0.02, 0.02	0.9
emo_anx	0.08 (0.43)	0.09 (0.41)	-0.01	-0.02, 0.00	0.017
emo_anger	0.14 (0.61)	0.13 (0.48)	0.01	0.00, 0.02	0.062
emo_sad	0.07 (0.46)	0.07 (0.38)	-0.01	-0.01, 0.00	0.2
1 Mean (SD)
2 Paired t-test
3 CI = Confidence Interval

Characteristic	5 wk, N = 21,9921	baseline, N = 21,9921	Difference2	95% CI2,3	p-value2
i	3.57 (3.53)	3.62 (3.00)	-0.04	-0.10, 0.01	0.2
we	0.55 (1.38)	0.50 (1.07)	0.04	0.02, 0.06	<0.001
they	1.39 (2.12)	1.41 (1.76)	-0.02	-0.05, 0.02	0.4
Analytic	48 (26)	48 (22)	0.09	-0.35, 0.52	0.7
cogproc	12.2 (5.8)	12.2 (4.9)	-0.03	-0.13, 0.07	0.5
prosocial	0.45 (1.13)	0.44 (0.96)	0.00	-0.02, 0.02	0.8
emo_anx	0.07 (0.44)	0.08 (0.37)	-0.01	-0.01, 0.00	0.11
emo_anger	0.14 (0.65)	0.13 (0.49)	0.01	0.00, 0.02	0.079
emo_sad	0.08 (0.49)	0.07 (0.35)	0.01	0.00, 0.01	0.15
1 Mean (SD)
2 Paired t-test
3 CI = Confidence Interval

Characteristic	6 wk, N = 19,9011	baseline, N = 19,9011	Difference2	95% CI2,3	p-value2
i	3.6 (3.5)	3.6 (3.0)	0.00	-0.06, 0.06	0.9
we	0.53 (1.37)	0.51 (1.05)	0.03	0.00, 0.05	0.027
they	1.40 (2.09)	1.38 (1.72)	0.02	-0.02, 0.06	0.3
Analytic	47 (27)	48 (22)	-0.33	-0.79, 0.12	0.2
cogproc	12.2 (6.0)	12.1 (4.9)	0.07	-0.03, 0.18	0.15
prosocial	0.44 (1.15)	0.45 (0.98)	-0.01	-0.03, 0.01	0.3
emo_anx	0.07 (0.43)	0.08 (0.40)	-0.01	-0.02, 0.00	0.004
emo_anger	0.14 (0.65)	0.13 (0.49)	0.01	0.00, 0.02	0.12
emo_sad	0.07 (0.45)	0.07 (0.37)	0.00	-0.01, 0.01	0.8
1 Mean (SD)
2 Paired t-test
3 CI = Confidence Interval

Disasters

Characteristic	-1 wk, N = 81,0611	baseline, N = 81,0611	Difference2	95% CI2,3	p-value2
i	3.59 (3.52)	3.59 (2.98)	0.00	-0.03, 0.02	0.8
we	0.54 (1.32)	0.55 (1.12)	-0.01	-0.03, 0.00	0.018
they	1.42 (2.15)	1.41 (1.74)	0.00	-0.02, 0.02	0.8
Analytic	47 (26)	47 (22)	-0.03	-0.25, 0.20	0.8
cogproc	12.3 (5.9)	12.3 (4.9)	0.03	-0.02, 0.08	0.2
prosocial	0.47 (1.16)	0.47 (0.98)	0.00	-0.01, 0.01	0.9
emo_anx	0.08 (0.46)	0.09 (0.39)	0.00	-0.01, 0.00	0.3
emo_anger	0.14 (0.63)	0.14 (0.50)	0.00	0.00, 0.01	0.5
emo_sad	0.08 (0.46)	0.07 (0.36)	0.00	0.00, 0.01	0.2
1 Mean (SD)
2 Paired t-test
3 CI = Confidence Interval

Characteristic	0 wk, N = 81,0861	baseline, N = 81,0861	Difference2	95% CI2,3	p-value2
i	3.6 (3.5)	3.6 (3.0)	0.02	-0.01, 0.05	0.14
we	0.57 (1.37)	0.54 (1.12)	0.02	0.01, 0.03	<0.001
they	1.40 (2.12)	1.40 (1.73)	-0.01	-0.03, 0.01	0.3
Analytic	47 (26)	47 (22)	-0.03	-0.26, 0.19	0.8
cogproc	12.2 (5.9)	12.3 (4.9)	-0.07	-0.12, -0.02	0.005
prosocial	0.46 (1.14)	0.46 (0.95)	0.00	-0.01, 0.01	0.3
emo_anx	0.09 (0.50)	0.09 (0.39)	0.01	0.00, 0.01	0.001
emo_anger	0.14 (0.64)	0.14 (0.50)	0.00	0.00, 0.01	0.083
emo_sad	0.08 (0.47)	0.08 (0.39)	0.00	0.00, 0.01	0.5
1 Mean (SD)
2 Paired t-test
3 CI = Confidence Interval

Characteristic	1 wk, N = 87,2931	baseline, N = 87,2931	Difference2	95% CI2,3	p-value2
i	3.6 (3.5)	3.6 (3.1)	-0.02	-0.05, 0.01	0.2
we	0.64 (1.46)	0.55 (1.16)	0.09	0.08, 0.10	<0.001
they	1.34 (2.05)	1.41 (1.77)	-0.06	-0.08, -0.05	<0.001
Analytic	47 (26)	47 (23)	0.24	0.03, 0.46	0.029
cogproc	12.1 (5.8)	12.2 (5.0)	-0.11	-0.15, -0.06	<0.001
prosocial	0.47 (1.17)	0.47 (1.00)	0.00	-0.01, 0.01	0.7
emo_anx	0.10 (0.50)	0.09 (0.41)	0.02	0.01, 0.02	<0.001
emo_anger	0.13 (0.58)	0.14 (0.52)	-0.01	-0.02, -0.01	<0.001
emo_sad	0.07 (0.44)	0.08 (0.39)	0.00	-0.01, 0.00	0.2
1 Mean (SD)
2 Paired t-test
3 CI = Confidence Interval

Characteristic	2 wk, N = 78,9051	baseline, N = 78,9051	Difference2	95% CI2,3	p-value2
i	3.57 (3.50)	3.59 (3.03)	-0.03	-0.06, 0.00	0.071
we	0.57 (1.38)	0.55 (1.14)	0.02	0.01, 0.04	<0.001
they	1.41 (2.10)	1.41 (1.77)	0.00	-0.02, 0.01	0.6
Analytic	47 (26)	47 (22)	0.24	0.02, 0.47	0.034
cogproc	12.2 (5.9)	12.3 (4.9)	-0.04	-0.09, 0.02	0.2
prosocial	0.47 (1.17)	0.46 (0.96)	0.01	0.00, 0.02	0.063
emo_anx	0.09 (0.47)	0.09 (0.39)	0.00	0.00, 0.00	0.8
emo_anger	0.14 (0.61)	0.14 (0.51)	0.00	-0.01, 0.00	0.2
emo_sad	0.08 (0.48)	0.08 (0.39)	0.00	0.00, 0.01	0.10
1 Mean (SD)
2 Paired t-test
3 CI = Confidence Interval

Characteristic	3 wk, N = 74,5211	baseline, N = 74,5211	Difference2	95% CI2,3	p-value2
i	3.6 (3.5)	3.6 (3.0)	0.00	-0.03, 0.03	0.9
we	0.55 (1.36)	0.54 (1.12)	0.01	0.00, 0.03	0.028
they	1.42 (2.12)	1.41 (1.75)	0.01	-0.01, 0.03	0.5
Analytic	47 (26)	47 (22)	-0.20	-0.43, 0.04	0.10
cogproc	12.3 (5.9)	12.2 (4.9)	0.01	-0.04, 0.07	0.6
prosocial	0.45 (1.15)	0.46 (0.97)	-0.01	-0.02, 0.00	0.2
emo_anx	0.09 (0.48)	0.09 (0.39)	0.00	0.00, 0.01	0.3
emo_anger	0.14 (0.62)	0.14 (0.51)	0.00	-0.01, 0.00	0.4
emo_sad	0.08 (0.47)	0.08 (0.38)	0.00	-0.01, 0.00	0.6
1 Mean (SD)
2 Paired t-test
3 CI = Confidence Interval

Characteristic	4 wk, N = 72,8871	baseline, N = 72,8871	Difference2	95% CI2,3	p-value2
i	3.57 (3.49)	3.58 (3.00)	-0.01	-0.04, 0.02	0.6
we	0.54 (1.32)	0.54 (1.12)	0.00	-0.02, 0.01	0.6
they	1.41 (2.12)	1.42 (1.75)	0.00	-0.02, 0.02	0.8
Analytic	47 (26)	47 (22)	-0.07	-0.31, 0.17	0.6
cogproc	12.3 (5.8)	12.3 (4.9)	0.00	-0.05, 0.05	0.9
prosocial	0.45 (1.14)	0.45 (0.95)	0.00	-0.01, 0.01	0.9
emo_anx	0.08 (0.46)	0.08 (0.38)	0.00	0.00, 0.00	0.9
emo_anger	0.14 (0.61)	0.14 (0.51)	0.00	0.00, 0.01	0.8
emo_sad	0.07 (0.44)	0.08 (0.39)	-0.01	-0.01, 0.00	0.004
1 Mean (SD)
2 Paired t-test
3 CI = Confidence Interval

Characteristic	5 wk, N = 72,7991	baseline, N = 72,7991	Difference2	95% CI2,3	p-value2
i	3.54 (3.48)	3.56 (3.00)	-0.03	-0.06, 0.00	0.091
we	0.55 (1.36)	0.54 (1.12)	0.01	0.00, 0.03	0.021
they	1.42 (2.12)	1.42 (1.74)	0.01	-0.01, 0.02	0.6
Analytic	47 (26)	47 (22)	0.23	-0.01, 0.46	0.061
cogproc	12.3 (5.9)	12.3 (4.9)	0.04	-0.01, 0.09	0.13
prosocial	0.46 (1.16)	0.45 (0.95)	0.01	0.00, 0.02	0.078
emo_anx	0.09 (0.47)	0.08 (0.38)	0.00	0.00, 0.01	0.5
emo_anger	0.14 (0.61)	0.14 (0.51)	0.00	0.00, 0.01	0.5
emo_sad	0.07 (0.45)	0.07 (0.38)	0.00	-0.01, 0.00	0.3
1 Mean (SD)
2 Paired t-test
3 CI = Confidence Interval

Characteristic	6 wk, N = 64,1561	baseline, N = 64,1561	Difference2	95% CI2,3	p-value2
i	3.56 (3.51)	3.56 (2.98)	0.00	-0.03, 0.04	0.8
we	0.55 (1.36)	0.54 (1.11)	0.01	0.00, 0.02	0.2
they	1.43 (2.14)	1.41 (1.71)	0.02	0.00, 0.04	0.035
Analytic	47 (26)	47 (22)	-0.21	-0.46, 0.04	0.10
cogproc	12.3 (5.9)	12.3 (4.9)	0.05	0.00, 0.11	0.060
prosocial	0.45 (1.13)	0.45 (0.94)	0.00	-0.01, 0.01	0.8
emo_anx	0.08 (0.46)	0.08 (0.37)	0.00	0.00, 0.01	0.8
emo_anger	0.14 (0.62)	0.14 (0.49)	0.01	0.00, 0.01	0.079
emo_sad	0.07 (0.45)	0.07 (0.38)	0.00	-0.01, 0.00	0.7
1 Mean (SD)
2 Paired t-test
3 CI = Confidence Interval

Can I see the graphs on your poster again?

Absolutely!